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Abstract. The fundamental mathematical definitions of the controlled feed- 
back Markov dynamics of quantum-mechanical systems are introduced with 
regard to the dynamical reduction and filtering of quantum states in the course 
of quantum measurement in either discrete or continuous real time. The con- 
cept of sufficient coordinates for the description of a posteriori quantum states 
in a given class is introduced and it is proved that they form a classical Markov 
process with values in either state operators or state vector space. The general 
problem of optimal control of a quantum-mechanical system is discussed and 
the corresponding Bellman equation in the space of sufficient coordinates is 
derived. The results are illustrated in the example of control of the semigroup 
dynamics of a quantum system that is instantaneously observed at discrete 
times and evolves between measurement times according to the Schrodinger 
equation. 



1. Introduction 

The encouraging outlook for the apphcation of coherent quantum optics (lasers) 
for communications and control has been recently stimulated by the steadily grow- 
ing demands for greater accuracy of observation and monitoring, particularly under 
the "extreme" conditions of very faint signals at extremely great (astronomical) dis- 
tances. On the other hand, instances of the successful exploitation of mathematical 
methods from information and control theory for the investigation of many physical 
phenomena in the microscopic world have also stimulated interest in the theoretical 
study, using general cybernetic principles, of the possibilities of dynamical systems 
described at the quantum-mechanical level 19 15 16jQ]0]. It has been shown 
in |5] that it is natural to regard many physical problems as control problems for 
distributed systems described by standard quantum- mechanical equations. In par- 
ticular, the possibility of the transition of a physical system from one microscopic 
state to another can be investigated [S] by the methods of the theory of control- 
lability on Lie groups generated by the Schrodinger equation with a controlled 
Hamiltonian. 

General problems in the theory of quantum dynamical systems with observation, 
control and feedback channels can be handled on the basis of the recent develop- 
ment of an operational theory of open-loop quantum systems, for which the 
mathematical formalism was set down in jlUj |13| . The investigation, undertaken 
in [3| , of the dynamical observation and feedback control optimization problems for 
such systems has provided a means for solving these problems in the case of linear 
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Markov systems of the boson type, in particular for a controllable and observable 
quantum oscillator 0]. This work was based on the multistage quantum-statistical 
decision theory originally described in for the problems of the optimal dy- 

namical measurement and control of classical (i.e., commutative) Markov processes 
with quantum observation channels. 

In the present article we describe a simplified problem of optimal feedback control 
of quantum dynamical systems which does not involve quantum-statistical decision 
theory. Here the observable subsystem at the output of the observable channel is 
regarded as classical and amendable to description at the macroscopic level, whereas 
the controlled entity remains a quantum dynamical system. In other words, we 
assume here, in contrast with that the "instrument" at the output of the 

quantum-mechanical system is given, rather than to be optimized, and it is required 
only to find the optimal macroscopic feedback for a given performance criteria. The 
results obtained in this setting are special in relation to 12 as they correspond 
to the semiclassical case of commutativity of the algebra of output observables. 
They nonetheless deserve special consideration both from the methodological and 
from the practical point of view when the observation channels are given and cannot 
be optimized for the optimal feedback control purpose. 

2. Controllable quantum dynamical systems with observation 

Here we introduce the mathematical concept of controllable quantum system 
with observation channel on the basis of the operational theory on open-loop phys- 
ical systems and quantum processes 1220] El- Such systems are open by the 
definition, and the necessary concepts borrowed from the algebraic theory of open 
quantum systems are described in the Appendix. 

Let Ti be the Hilbert space of representation of a certain quantum-mechanical 
system regarded as an observable and controllable system and let 21 be the von 
Neumann algebra of admissible physical quantities Q S 21 which is generated (see 
Appendix 1) by the dynamical variables of this system, acting as operators in 7i. 
The pair {7i,2l} plays the role of a measurable space {X, ^} representing jj^ the 
corresponding classical dynamical system in the phase space X of it's point states, 
endowed with the Borel a-algebra A of admissible events A ^ A. The simple 
systems normally treated in traditional texts on quantum mechanics, for example 
[T7j . correspond to the algebras 2t = B {H) of all bounded operators in H, but the 
models that emerge from quantum field theory and statistical mechanics |12| are 
described by the more general algebras 21. 

Normal states of the quantum-mechanical system at every time i 6 R are de- 
termined by the linear functionals gt '■ Q '—^ {pt,Q) of the quantum-mechanical 
expectations {Q)^ = (pt.Q) of all the physical quantities Q G 2t for this system, 
and are described by the densities pt as the positive elements associated with von 
Neumann algebra 2t (see Appendix 2). In the case of semifinite algebras as in 
the simple case 21 = B{H), the states gt are usually represented by the trace one 
operators pt in 21 (or affiliated with 21) as 

(2.1) gtiQ)^tr{ptQ} = {puQ). 

The Master evolution 1 gt oi a quantum-mechanical system controlled on each 
time interval [t, t+r) by a segment uj = {it (r) : r G [t, t + r)} of certain parameters 
u (t) is usually described by linear unital completely positive transformations of the 
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states 

(2.2) ft e a. ^ gtMl (ul) = Qt+r {uD e a. Vi e M, r > 0. 

They are determined by the composition of the functionals : 21 i-^ C with con- 
trolled transfer operators M[ (u[) as the maps 2t 2t defined in the Appendix 
3. The Markov family {MJ'jjgg of these maps must satisfy the consistency 
condition 

(2.3) M; «) M;+^ (<+^) = M;+^' Vr e M, r, r' > 0, 

as quantum analog of the controlled Chapman-Kolmogorov equation. Here u^. = 
{u [r + t')}^,^^ is a segment of an admissible control function u{t) £ U (t) of length 

r > and u^^'^ denotes the composition (^u^, ul_^^^ of segments and u'^._^_^. Note 

that for stationary (time shift-invariant) systems, where U (t) = U, (|2.3|) specifies 
the cocycle conditions for transfer operators M"^ (u^) ~ (u^) independent of r 
with respect to the shift u'^ i— > given by Ur (t) = u{t + r). 

In the spatial case of transfer operators MJ" {uj) specified by controllable prop- 
agators T {uj) :Ti^ Ti satisfying the condition corresponding to H2.2|l . 

t;;, t; «) = r;+-' , 

the dynamics Qt ^ Qt+r (uf) for the vector states {pt,Q) — {''Pt\Q4't) is described 
by the state-vector transformations 

^t&n^ Tl «) Vt = V't+r «) G W Vt e R, r > 0. 

Such transformations can be obtained, for example, as the fundamental solutions 
of the time-dependant Schrodinger equation with a perturbing force u (i) , for which 
the isometric operators Tl {uj) are unitary, and MJ {uj) are the Heisenberg trans- 
formations 

M[ «)Q = T;«)tQT«). 

The open-loop input-output quantum dynamical systems of the kind specified 
below as controllable systems with observation cannot, as a rule, be described in 
terms of propagators T[ (ul), because the measurements induce reductions and de- 
coherence of quantum states which are described by more general transformations. 
In order to define such systems as quantum dynamical objects with the classical 
inputs u (t) and outputs v (t) G V (t) , we shall fix a pair of two-parameter families 
{U^} , {V^} , i e R, T > of the Cartesian product subsets U[ C U U {t + r'), 

r'<r 

Vfl = Y[ ^ (^ + ''"'); satisfying the consistency condition 

r'<T 

(2.4) t/; X [/;+^ = c/;+^', v; x = v;+^'. 

The elements uj G Ul ^vj e VI are called the admissible segments of signals, and 
their Cartesian compositions should also be admissible in order to make sense, 
for example, of (|2.3|l . The sets Ul are usually endowed with Hausdorff product 
topologies, and the sets VI with Borel product tr-algebras, making them consistent 
with 1)2. 4|) . Note that the continuous signals u {t) and v {t) on t gM. do not satisfy 
the conditions (|2.4|l . In the time shift-invariant case Ul and VI are given by the 
shift 1 1^ t + r of the initial sets U^ = m and V^ = VJ . 
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Definition 1. A controllable quantum dynamical system with observation is de- 
scribed by a family {HJ'ljgjj of controllable transfer- operator measures HI (uj", dyj) : 
21 (—> 21 (see Appendix 4) defined on the spaces U[ 3 uJ^Vf^ ^ dvj and satisfying 
the condition 

(2.5) n; {ui,dvi) n;^^ = n;+^' 

for any r G R, r, r' > 0, where u^^"^ = (^ul^u^^^^, dv^^^ = dv^. x dvj.^^. 

The superoperators HJ" (uj", dvl) are assumed to be continuous in uj", cr-additive 
with respect to dvl (ii^ the strong operator sense) and in the time shift-invariant 
case not to depend exphcitly on t [the index t in the condition (|2.5|l specifying the 
cocycle dependence on t can now be omitted] . 

Due to positivity Q G 2t+ > IIJ" (mJ', du^) Q G and the normahzation condi- 
tion nj" (u1, V^) = MJ" (mJ^), the mappings IIJ' determine, for a given instantaneous 
state Qt G 2t* and control function m[, the future (r > 0) states 



(2.6) Qt+r {ul,dvj) = etn[ «, dvl) 

of the quantum-mechanical process, normalized to the probabilities 

(2.7) < {uhdvB = (pt,n[ = gUl {ul,dvl)I 

of the events vl € dvj . The ratio of (|2.6|l to (|2.7|l determines conditional states, 
depending in general non-linearly on g = gt'. 

/ooA „T 9^1 (ul, dvl) 



gni {ul,dvl)I 



Note that the normalized conditional states are well-defined only for the measurable 
events dvl ^ of non-zero probability H2.7() with which the system transfers from 
the state gt — g as & result of the control action ul and the observation dvl "^'^ 
interval of length r. If the event dvl — is certain, ul [ul, dvl) ~ 1; the states 
(|2.8|) are unconditional such that gl coincide with a priori states gt+r ifJ-t)j whose 
controlled evolution is linear. Such evolution is described by expression H2.2|l . 
in which M[ (u[) — III (""t"; ^1) denotes controllable transforms of the open-loop 
quantum-mechanical system, corresponding to the absence of observation. However 
in general the process of precise measurement of the output signal vl on an interval 
of length r > takes the quantum system from a priori state g = gt to the a pos- 
teriori state gl = (?M^ ^ {ul , vl), where the quasilinear mapping g i— > (?M^ ^ {ul , vl) 
is given by expression H2.8|l in the limit dvl i {^1} almost everywhere with respect 
to the measure (|2.7f) . For example, let the measures HI have the density functions 

(2.9) Ul {ul, dvl) = JPl i^hvl) fil {ul, dvl) , 

where PJ" {ul, vl) : 21 1— > 21 denotes completely positive superoperators [see the Ap- 
pendix, (A. 4)], say, of the form H3.10|l . continuous with respect to u[ and integrable 
with respect to vl in the strong operator sense with respect to specified positive 



QUANTUM FEEDBACK CONTROL 



5 



measures /ij" on Vf^ . Then the a posteriori transfer operators ^ (uj", vj) coincide, 
up to normalization, with PJ" (uj", v^) : 

where the ratio is defined for those u[ e , vj G Vj'^ for which the densities 

(2.11) pj «, vl) = (p, P[ «, ^;[) /) ^ eP[ «, <) / 

of the probabihty measure H2.7|l with respect to pi are non- vanishing. 

The foUowing theorem states that the a posteriori mapping H2.10|l in fact de- 
termines the state-vahied classical Markov process, which was introduced in the 
classical case by Stratonovich in |20j (He called this probability measure-valued 
process secondary, or conditional (a posteriori) Markov process). 

Theorem 1. The family jMp^} of a posteriori transfer operators Mp f. (uj ,vl) 
satisfies, with respect to the operator composition, the consistency condition 

(2.12) m;, «, <) m;,;,+, = m;^' 

almost everywhere under the measure \2//{ l, where p' is the density of g' = gMp r ("ri ) 

Proof. It is required to verify the property (|2.12l) for conditional mappings H2.8|l . 
for which it follows at once from the definition and 1)2. 5|) . Then it applies also in the 
single point limit dvj | {vj}. In the case (|2.9() the condition H2.12|l is simply verified 
by computing the product H2.12|l of the a posteriori transfer operators H2.10|l : for 
this purpose it is necessary to invoke the corresponding condition 

(2.13) P; «, <) P,V (<+r, Vr+r) ^ K^^' ■ 

It is sufficient to require this composition condition for Vf^^"^ almost everywhere 
(mod/x[) and this will guarantees the satisfaction of condition l|2.4|l if 



/ij^ (ul, dvl) /i^+T- {u^^T, rfVr+r) = A* {y'r^^ ' dv^^'^ j 



□ 



Remark 1. If the superoperator densities PJ" of the transition measures Y2.y\) pre- 
serve unity: PJ" (uj", v^) I = pj (uj", vj) I , the ratio I^<1.1U\) determines p-independent 
transfer operators (u^ ,vl) describing the controllable quantum dynamics of a 
system with two inputs u and v. The second is an observable stochastic process 
with probability measures nj {ul,vl) = p[ (wj", u^) /ij" (uj", wj") independent of the 
state of the system. The a posteriori mappings 1^2. 8\) in this case are linear, 
qI = {ul,vj), almost everywhere under the measure nj . 

3. Sufficient coordinates of quantum-mechanical systems 

The description of the dynamics of simple closed-loop quantum-mechanical sys- 
tems for a certain class of initial states is known to be often reducible to the determi- 
nation of the time evolution of certain coordinates, the role of which can be taken, 
for example, by vectors -0 G 7i, if only vector initial states are considered. Some 
aspects of the controllability of closed-loop quantum-mechanical systems described 
by a sufficient coordinate V't G satisfying the controlled Schrodinger equation 
have been recently investigated in 
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The concept of sufficient coordinates, wtiich is introduced below for general con- 
trollable quantum dynamical systems with observation and is intimately related 
to the classical notion of sufficient statistics 20 , plays an even greater role for 
quantum control theory than the analogous concept in stochastic control theory, 
because it permits control problems for quantum-mechanical systems to be reduced 
to classical control problems with localized or distributed parameters. 

Definition 2. Let X be a measurable space^, and let {Qx,t}j.^x tev. a family of 
states given, for every i S R, by a measurable mapping x i—s- g^^t of the space X 
into the space of states Qx.t £ 21^, of a quantum- mechanical system at time t such 
that the controlled evolution 1^2. 5\) of the system during an observation leaves this 
family up to a normalization t:], j invariant: 

(3-1) &,tn[ {ul,dvl) = «, dvl) Qfr ^(ul.vl).t+T 

Then x E X is called a sufficient coordinate for {Qx,t\, the controlled stochastic 
evolution of which x <E X ^ /^^ {ul,vl) = xl E X is described by the mappings 
f^f : UJ X — > X, continuous with respect to G UJ and measurable with 
respect to vl e V^ almost everywhere under the measure 

7TlAuldvl) = {px,Uj iuj,dvj)l}. 

Proceeding from (|3.f |l taken in the limit dvj \ {vf}, we note that the density 
operators x — p form the sufficient coordinate space for the family of all normal 
states Q on 21. It is given by the a posteriori mapping /J^ (mJ", vl) — j (mJ", vf.)^ p, 
provided only [as in the case (|2.9|) ] that there exists the derivative 

as the limit dvj \ {vj}. 

Theorem 2. The mappings f^t J.*?. l\l define a sufficient statistics xl — fx t (■"Ij 
for Qt G {Ox.tlxex sense that the a posteriori states qI — f?M^ ^ {ul,v1) are 

determined for such g = Qx.t as = Qxi,t+T^''' > 0- Moreover, the transition 
probabilities 

(3.3) <,«,dx') = (p.,n;,«,dx')), 

from Xt — X into dx' 3 xl , defined by 

n;, {ui,dx') = n[ {ui, f-} «, dx')) , 



(3.4) f-^t iuj,dx') = {v; : «, vl) e dx'} , 
satisfy the C'hapman-Kolmogorov equation 

(3.5) I «, dx') dx") = <+^' (u;+^',dx") 

x'GX 

for all r G M, T, t' > and mJ[+'^ = (^u^. , u^^^) , so that the sufficient statistics 
form a controllable Markov process. 

^For all practical purpoccs it is always sufficient to assume that X is a standard Borel space, 
i.e. a complete seperable metric space, also known as a Polish space (for example, R", C", or any 
countable set). 
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Proof. The existence of the sufficient statistics is determined by the a posteriori 
mapping according to the expression (|2.8|l . which gives in correspondence with 

(3-6) gl = gMg^t «, = gf-,{u-,vj),t+T' 

for g = gx,t, thus proving the first statement of Theorem [3 The transition map- 
pings X I— > f (ul,vl) in correspondence with the Theorem ^ satisfy the semigroup 
property with respect to their composition 

(3.7) o /; {u;, <) = . 

This yields, according to I2.5|l . the equation 

(3.8) J Ul^, «, dx') Ut.,r+r ifr.dx") = {ul+^\ dx" 

in terms of the transfer-operator measures for the transitions x i— » dx' specified 
in 1)3. 3|l and (|3.4|) . For the states in the class {gx,t], H3.8|) is equivalent to the 
Chapman-Kolmogorov equation H3.5|l . as 

(3.9) gx,t^l,t «j dx') = ttIj. {ul,dx') g^'.t+r 

in accordance with (|3.1|) . Thus equation (|3.5|) determines a Markov stochastic 
evolution x{t) of the sufficient coordinates x (t) G X, which is described, according 
to the main Kolmogorov theorem (see, e.g. [TUI, p. 48 for a standard space X), a 
Markov probability measure in the functional Borel space of the trajectories {x (t)}. 
This completes the proof. □ 

We now discuss in more detail the special case, in which the transition measures 
(|2.9|) have the superoperator densities 

(3.10) P[ «,<)Q = {ul,vl)^QF^ (ulvl) 

with respect to a given consistent family of measures /ij" (uj', dvj). Here the oper- 
ators Fl {ul,vj) are assumed to satisfy the normalization condition 

(3.11) J F; «, <)t Fl «, vj) «, dvl) = /, 
in the Hilbert space Ti., as well as the composition condition 

(3.12) Fj,_f^^ [uj.^^,Vf.^^] Fj. {u^,v.j,) = Fj.'^ 



which guarantees the fulfillment of H2.13|l . 

One can easily see that the a posteriori transfer operators (|2.1U|) preserve the 
vectorial property of the states g^, {Q) ~ {tp\Qip) such that 

4,* {Q)^{i^imi) = g^^r{Q), 

where ipl — ^ (uj", v^) ip for any unit vector ip £ Ti, t > and 

Corollary 1. The normalized vectors € Ti.,\\ip\\ = 1 in the case \3.1U\) form 
the space X of sufficient coordinates x = ip specified by the a posteriori mappings 
f^ ( (ii[, vJ) — I (u[, vf)il) of the quasilinear form 1^3. 1S\) for the family of all 
vectorial states g^. 
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We note that the a posteriori propagators ^ {u^ ,vl) satisfy the semigroup 
property 

(3.14) t;,;,+, t^^ O = t;+/ , 

where ■0' = (ul,v^)^p. They are nonHnear (quasiUnear) , and in contrast with 
the Hnear operators F[ (wj", vj) they preserve the norm in Ti.. Only in the case dis- 
cussed at the end of Section|3| where (it[, vJ)'' F^ {uj, vl) — pj {uj, vJ) I, the op- 
erators (|3.13|l are ^/i- independent isometrics {uj, vl) — F^ {uj, vl) j \Jp'[ (uj , vl). 
We note, however, that the a priori transfer operators 

(3.15) M[ «) g = Jf[ «, vl)^ QF «, vl) /i[ «, dvl) 

determining the controUable Markov dynamics of the quantum system (|2.9f) . (I3.10f) 
in the absence of observations, are not described by the propagators (ul), with 
the exception of the degenerate case in which the a posteriori states coincide mod /ij" 
with a priori states, i.e., actually do not depend on the results of the observations 
vl- 



4. Optimal quantum feedback control 

Let us now discuss the optimal control of a quantum dynamical system with ob- 
servation {n[}. We assume that the performance of the system is measured at each 
time t by the mathematical expectation {pt, Qt {ut, dvt)) of a certain physical quan- 
tity Qt{ut,dvt) G 21 which continuously depends in strong operator topology on 
the input state Ut = {u{t + r)}^^^ and on the output event dvt = d{v [t -\- t)}^^q 
according to the equation 

(4.1) Qt {ut,dvt) = n[ (u[, dvl) Qt+r (ut+T-.dvt+r) + SI {ul,dvl) . 

Here SI [ul,dvl) G 21 are Hermitian operators having the integral form 

r 

(4,2, srw,*n^/nr(.-r,*r)s(..(«+/).«+/)^.' 



for a Hermitian operator-function S {u,t) = S'(u,t)^ completely determining (|4.1(l 
for a certain boundary condition Qt {uT,dvT) = g at the final time T > t. The 
conditions for the existence of the integral 14.21 its continuous dependence on m[, 
and its cr-additivity with respect to dvl, requiring the continuity in u G U and 
measurability in t G K for the operator function (u, t) i-^ S {t, u) G 2t under strong 
operator topology, are presumed to be fulfilled. The operator Q, specifying the 
terminal risk {pt,Q), is assumed to be Hermitian-positive. 

Definition 3. A measurable mapping Vt i— > Ut (vt) G Ut is called a non- anticipating 
control strategy if its components u{t + t, ■) : Vt ^ u[t ~\- t) are determined by func- 
tions independent ofvt+r- It is called a retarded control strategy if all u {t + t, ■) are 

determined by functions vl t-^ u (^t + t,vI ^ for some measurable t' = t' {t + t) < 

T. A non- anticipating strategy Ut (■) is called admissible if the integral 

Qt [u (•)] ^ I Qt [ut {vt) , dvt) 



QUANTUM FEEDBACK CONTROL 



9 



exists in strong operator topology, and it is called optimal for an initial state Qt — Q 
if it realizes the extremum 

(4.3) q{p,t)= inf (p,Qi[ui (.)]), 

ut(-)eUt(-) 

where Ut (•) is a certain set of admissible strategies ut (•) [e-optimal if (p, Qt [ut (•)]} 
exceeds at most by e] . 

We note that in accordance with 14.1|l . a strategy ut (•) is admissible with respect 
to Qt (•, •) if and only if its segments ut+r (■) are admissible strategies with respect 
to Qt+T ■) for each fixed vj , and if there exists measure 

(4.4) ^ridvl)^ I Uliul{vl),dvB, 
specifying the operator-valued integral 

r 

(4.5) SUuU-)]^! Jnr' {dvl')s[t + T\u{t + T',vl')) 

V- 

for each strategy segment {■). The latter holds for any delayed strategy that is 
admissible for a given boundary condition Qt (■, ■) = Q- 

Theorem 3. Let the sets U[ {■) of admissible strategy segments satisfy the condition 

(4.6) U[i-)xU[iA-)^Ul+^' {■) VteM, r,T'>0. 

Then the minimal risk as a function of the density operator p and the time t 

satisfies the functional equation 

{p, SI [ul (.)]) + I 7r;-J {dvD q {p [vD ,t + T) , 

where n^'l (■) — (p, 11"'^ (.) /), p (vj) = j (uj" (vj) , vj)^ p denotes the probability 
measures {2.7)) and a posteriori states \2.^) corresponding to an admissible strategy 
u = ul {■) and an initial state Q = Qt- 

Proof. The proof of 14.7|l generalizes the proof of the Bellman equation [7j. By sub- 
stitution of H4.1|l into (|4.3|l it reduces the minimization over ut {■) by the successive 
minimization of (|4.7|) . first on ut+T (■) and then on ul {■), which by condition (|4.t)|) 
yields the same result as (|4.3ll . Since the integral H4.5|) does not depend on ut+r (■) 
and by definition, 

(4.8) elir{dvl) = 7:ll{dvl)g;^j, 

the first minimization entails finding the second term of the minimized sum H4.7II : 

«t+ (•)ei^t+ (■) / ^^^"^ ' / ' '^"*'*) " / '^^^'^ ^ ''^ '^^^^ ' ^ + 'r) • 

□ 

In the case of a given boundary condition q (p, t) = (p, Q) the theorem proved 
above provides a constructive method of synthesizing an optimal or e-optimal strat- 
egy ujj* (^?'~*) by the successive minimization of (|4.7|) in reverse time. In this 



(4.7) q{p,t)^ inf 
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case it is sufficient to restrict the discussion to Markov admissible strategies de- 
scribed by segments u'^^t (vj,), t = T — t, depending on the a priori history 

only through the agency of their dependence on the a posteriori state g = gl~* 
for any t' > t. Accordingly, the determination of the a posteriori quantum states 
gj, which generate the a posteriori Markov process, enables us to reduce the opti- 
mal quantum control problem to the classical problem of stochastic control theory 
|2()j 13 with usual transition probabilities and final risk functions 

s (p, t, u) ^ (p, S (t, u)) , q {p, T) = (p, Q) , 

determined by the operators of the corresponding quantum variables S {t, u) and 
Q. 

Let us consider the case in which the quantum states g are considered in a certain 
class {gx,t} for which sufficient coordinates exist. 

Corollary 2. Let fl : U[ x V^^ i-^ X denote mappings satisfying the conditions 
of Theorem\^ Then in problem i4.3{ ) for g g {&,*} it is sufficient to restrict the 
discussion to Markov strategies described by measurable mappings ul : Xx Vi ^ 
satisfying the consistency condition 

(4.9) (ul^ «) , = ' 

where x' ~ f^'l (vf) — f^^ {uj (vf) ,vf). In particular, the instantaneous control 
functions (t) for any t G [t, T) arc determined by functions u (r, x) of the point 
state X in accordance with the equation 

(4.10) ux {t + T, vl) = u{t + T, (vj)) . 

The foregoing assertion, which follows directly for the "maximum" sufficient 
coordinate x (t) = p (t) from the optimality equation H4.7|) , is readily proved on the 
basis of the properties formulated for sufficient coordinates in Theorem |21 

The further simplification of problem H4.3|) entails utilizing the specific properties 
of the Markov process x (t) = /^'g (wq), the role of which is logically assigned to 
sufficient coordinates of the fewest possible dimensions. 

For example, in the case where the generator for the transition probabilities 
7r!J! ( {dx'), defined as the t-continuous strong limit 

(4.11) L (t) q (t) [x] =. lim / dx') {q {x' , t + t) - q {x,t + t)) , 

X 

exists on some set V {X) of bounded and measurable functions x t-^ q {x,t), depend- 
ing continuously on t, the optimality equation H4.3|) is written in the infinitesimal 
form 

(4.12) — I? {x, t) = inf {s [x, t,u) + L {t, u) q (t) (x)} , 

ot ueu 

where s(x,t,u) = {pxt, S (t,u)). Equation (|4.7|l . which represents the standard 
Bellman equation for controlled Markov processes in continuous time, can be used, 
together with a boundary condition q{x,T) = {px,T:Q) € V{X), to seek opti- 
mal or e-optimal Markov control functions u (t) directly as functions u {t, x) of the 
instantaneous state x. 
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5. Quantum control with discrete observation 

As an example here we consider the controhed dynamics of a simple quantum 
system described between discrete measurement times T = {tk] by the Schrodinger 
equation 

(5.1) ih^^^pit)=H{t,uit))^{t), teT. 

Here H{t,u) is the controlled Hamiltonian, i.e., a self-adjoint operator in Ti. with 
a dense domain of definition V CTi, written in the usual form 

m 

H (t, u) = Ho {t) + 5Z u» it) H, {t) , 

i=l 

where Ui (t) G M; Hi (t) are simple^ functions of t. Under the stated assump- 
tion there exists a unique consistent family {T^} of unitary propagators T[ (m[), 
representing for any ^ (t — t) = G 2?, a solution of (|5.1|l between adjacent mea- 
surement times tk < ifc+i in the form -0 (t) = T^_^ {^I-t) V": ^ G [tk,tk+i), where 
limi/) (t) = ip. 

Let Ey_k denote Hermitian projectors, which determine orthogonal decomposi- 
tions I — ^ of the unit operator in Ti. and specify measurements at times tk 

veVk 

of quantum physical quantities described by self-adjoint operators 

(5.2) ^k=J2 ^^-'fc 

with discrete spectra Vk C R. 

As a result of measurement of the quantity Ak there occurs a reduction JH| of 
the quantum state, g i— > gllv^k, v G Vk, described by the superoperators Ilv,kQ = 
Ev,kQEv^k, which determines a priori transfer operators 

(5.3) HfeQ — ^ Ey^kQEy^k- 

v£Vk 

The states gy_k = g^v,k to which the system transfers instantaneously depending 
on the result of this measurement v € Vk are normalized to the probabilities TTy^k = 
{p,Ey^k) of these transitions, where if g — g^i, is a vector state g^ (Q) = {^\Qtp), 
the states gy^k are also vectorial, determined by the projections ipv,k = Ey^k'4'- 
The product Ey^kT^ {uD — ^ (uj') for t — tk — t determines a transformation 
■0 (t) I— » Ey^kip (tk) corresponding to the evolution (|5.1|) on the interval [t,tk) with 
subsequent measurement of the quantity Ak. 

We introduce the notation Fy,k {uk) = Fy\ where Tk ~ tk+i — tk, and we 

set V^ = Yi ^k, where — {k : tk £ [t,t + t)} is the set of all indices of times 

kGK- 

in the interval (t, t + t) (in the case of an empty set = we assume that Vf^ 
consists of some single point {w}). 

Proposition 1. Let the set K^"^ be finite for any t < s. Then the chronological 
product 

(5.4) f:^' = rr*^ Kr*o ^-i.'-i ("'-i) • ■ • ^-.+1,'^ K) , 

^In other words, having one-sided limits. For unbounded self-adjoint operators Hi {t), i = 
0, m, this means that H (t, u (t)) i/i is a simple function for any ip £ T). 



12 



V. P. BELAVKIN 



where k = minii'j'' I ~ maxK^ *, and v — {vk, ■ ■ ■ ,vi) ~ v't *, determines 
controllable quantum dynamical system described by superoperators {Til} '^f 
form 113), 1223); 

(5.5) n;, «) Q = f;, K)t qf;, «) , 

under the counting measure fi^ — 1 on 3 v. 

The proof is the verification of conditions and (|3.12|l , which take the form 

(5.6) Kt «)^ Kt «) = ^ v< e ui, 

(5.7) (t.;;,) «) = ' 

where S V^_^^, v € . They are easily verified by induction, owing to the 
finiteness of the product (4.4). 

Because of the spatial form (|5.5|l of the consistent family {Il^.f}, on the basis of 
Corollary 1 we infer that the space X of normalized vectors ■(/; G ||'0|| = 1 forms 
a space of sufficient coordinates, the a posteriori evolution tp i— > ^ [ul^vDip of 
which is described by the nonlinear propagators (|3.13() : ^ {uD / ^ {uD ip^^ 
and the a priori evolution by transfer operators of the form (|3.15() : 

n[«)Q= E Fl,{ul)^QFl,{ul). 

vev- 

We give special consideration to the case of complete measurements described 
by the operators Ak with a non-degenerate spectrum. 

Proposition 2. Let {i^v,k\^^y^ denote the complete orthonormal systems of eigen- 
vectors of the operators A^, and let E^^k be the corresponding one- dimensional pro- 
jectors onto '4'v,k- Then the a posteriori states at the times {tk} are vector states, 
which are completely determined by the last result of measurement Vk G Vk '■ 

(5.8) (Pt""*, Q) = {Qipvk,k,tpvk.k) Vt <tk, g= Qt, 

and the measurement process {wfe} is a Markov process, which is described by the 
controllable transition probabilities 

(5.9) TTv^k {uk,Vk) = \{lpVk.k+l\Tk {Uk)'ipvk,k)\ ^, 

where Tk (uk) = T[^ {uH) , Tk = ife+i - tk- 
This proposition follows from the property 

(5.10) E^^kQEy^k = {'4'v,k\Qi'v,k) Ey^k 

of the one-dimensional orthogonal projection operators Ey k corresponding to the 
eigenvectors ipy^k, so that the application of any state g to IjS.Sfl dX t — tk ~ t leads 
to (|5.8|l . up to normalization. Since the a posteriori state (|5.8f) does not depend on 
the previous measurements, the conditional probability given by expression (|5.9|) 
for the event '^fc+i — v and fixed preceding results is Markovian. 

In the proposition proved above, the controllable sufficient coordinate Xk = 
Vk can be used, provided only that the quantum system is analyzed at discrete 
measurement times {tk} ■ 
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We now consider the optimal control problem for a discretely observed quantum 
system. Let the control performance, as a function of the initial t, be described by 
an operator (|4.1|l . which is determined by the integral H4.2|l of some operator- valued 
function S (t,u) : 7i i— > Ti.. 

Proposition 3. Under the conditions of Theorem\^ for a vector initial state 
the minimal risk 

(5.11) q i^P, t)= inf ^ / (VlQt (ut (vt) , dvt) V) 

in the intervals {tk,tk+i) between measurements satisfies the functional equation in 
variational derivatives 

(5.12) -^g i^P, t) = Jnf + 2h-^ Im {5q (^, t) /S^ {t, u) V-)) , 

where Wiplls — {^\'^^)- measurement time instances {t^} it satisfies the 

recursive equation 



(5.13) qk W = Jni^ Ufs^u,) + E <k W («)) : 

\ v&Vk+i ) 

which determines the boundary values q (tk — 0, V-") = Qk (^) for ^5.1^) . Here 
K,k W = (uk) H\ i> [v] = TvM {uk) i^l^K^)' 

T 

(5.14) Sk{uu) = j Tl;'" {u\l'-)^ S{t,u{t))Tl;''' (u\-'-)dt. 



Equation H5.12|l is readily proved on the assumption of analyticity of the function 
ip I— > q{ip,t) which is natural for a quadratic boundary condition q^ipjt) = HV'IIq 
at some final time T. Here (|5.12ll represents a functional version of the Bellman 
equation corresponding to the Schrodinger equation (|5.1|l and a quadratic transition 
cost function S {t, u, ip) ~ ||V-'lls(t u)- Equation H5.13|l follows directly from H4.7|l for 
t = tk, T — tk+i — tk and g = if it is taken into account that the integral H4.2(l 
now has the form (|5.14(l . 

In conclusion we consider the optimal control problem described above in the 
complete measurement case. Making use of the fact that the process of complete 
measurement at discrete times {tk} induces a Markov sufficient coordinate Xk — Vk, 
from ()5.13|l we deduce the customary equation 

(5.15) q,,{vk)^ inf \Sk{uk,Vk)+ TTv,k iuk,Vk) qk+i (v) 

which describes the optimum risk for the control of a discrete Markov process {wfe} 
with the transition probabilities (|5.9|) . a cost function Sfe(ufc,ffe) — 11^1)1-115^.(1,^) 
and a boundary condition of the form qk iv) — llV'^fcllg- The solution of derived 
Bellman equation H5.15|l can be easily modelled on a computer by standard dynamic 
programming methods for the piecewise-constant admissible strategies, for which 
C/fe = C/(<fe)CM'". 
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Appendix A. Notations, Definitions and Facts 

(1) Let {QOiei be a family of self-adjoint operators acting in a complex Hilbert 
space Ti. The von Neumann algebra generated by the family [Qi] is de- 
fined as the minimal weakly closed self-adjoint sub-algebra 21 of bounded 
operators in Ti containing the spectral projectors of this operators, along 
with the unit operator /. It consists of all bounded operators that com- 
mute with the commutant {Qi\' = {Q : QQi — QiQ Vi G I}, i.e., it is the 
second commutant ^ — {Qi} of the family {Qi}. The latter can be taken 
as the definition of the von Neumann algebra generated by the family {Qi} 
in the case of unbounded self-adjoint operators Qi densely defined on a 
domain V C 7i. The simplest example of von Neumann algebra is the 
algebra B {H) of all bounded operators acting in H |1T|. 

(2) A state on a von Neumann algebra 21 is defined as a linear ultraweakly 
continuous functional g : 21 ^ C (which will be denoted as g [Q) — {p, Q)) 
satisfying the positivity and normalization conditions 

(A.l) (p,Q)>0, VQ^O, (p,/) = l 

[Q ^ signifies the nonnegative definiteness (t/iIQ-j/') ^ V-!/) € H called 
Hermitian positivity of Q]. It is described by the density operators p as 
the elements of the algebra 21 with respect to a standard pairing {p,Q). 
The linear span of all states on 21 is a Banach subspace 2t+ of the dual 
space 21*, called predual to 21 as 2t* = 21. A state g is called vector state if 
{p,Q) = (i/'IQV') {q = Qi>) for some tp £ H. Any state is a closed convex 
hull of vector states g^, ||'0|| = 1. If on an algebra 21 there exists a normal 
semi-finite trace Q i— > tr{Q}, then the states on 21 can be described by 
the density operators p G 2t (or affiliated to 21, if they are unbounded), 
determining g by means of the bilinear form (p, Q) = tr {pQ}. For the 
case 21 = K (W) the density operator p is any nuclear positive operator with 
unit trace [TT] . 

(3) Let 2li, 2I2 be von Neumann algebras in respective Hilbert spaces TCi and 
7^2, and let M : 2t2 ^ 2li be a linear map that transforms the operators 
Q2 £ 2t2 into operators Qi £ 2li (superoperator, in the terminology of 
|13p. We shall call the operator M a transfer operator if it is ultraweakly 
continuous, completely positive in the sense 

(A.2) ^ ^7A,|M(Q|Qfc) Vfe) ^0, VQ, e2l2, G Hi, 

i,k=l 

{i = l,...,n < 00), and unity-preserving: M/2 = Ii- In this case the 
composition giM with a state fpi : 2li — > C is a (normal) state 2I2 ^ C 
described by the predual action of the superoperator M on pi: 

(pi,Mg2) = (M,pi,Q2),Vg2 e2l2,pi e2t\. 

A transfer operator M is called spatial if 

(A.3) MQ2 - T^Q2T or M,pi = TpiT\ 

where T : Hi ^ 7^2 is a linear isometric operator, T^T = /, called the 
propagator, and is the adjoint operator. Every transfer operator on 
2I2 = B (7^2) has a decomposition as a closed convex hull of spatial transfer 
operators. 
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(4) Let be a measurable space, and B its Borel cr-algebra. A mapping 11 : 
dv E B Il{dv) with values Il{dv) in ultraweakly continuous, completely 
positive superoperators 2I2 —>■ 2li is called a transfer-operator measure if 
for any pi G 2li, Q2 G 2I2 the numerical function 

{U{dv)^Pi,Q2) = {pi,n{dv) Q2) 

of the set dv <Z V is a countably additive measure normalized to unity for 
Q2 ~ I. In other words, 11 {dv) is an operator-valued measure that is 
(T-additive in the weak (strong) operator sense and for dv = V \s equal to 
some transfer operator M. The quantum-state transformations pi 1— >• p2 
corresponding to ideal measurements are described by transfer-operator 
measures of the form 

(A.4) n{dv)Q = F{v)^ QF{v)p{dv), 

where F {v) denotes linear operators Tii ~> H2, the integral under a positive 
numerical measure on is interpreted in strong operator topology, and 
J F'^ (v) F (v) p (dv) — Ii. Every transfer-operator M : 2t2 ^ 2li for 2I2 = 
B (Ti.) can be represented by the integral ljA.4|) with respect to dv C V of 
some ideal measure Il{dv). 
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